Conversation
…report structure Add methodology enforcement across the sigint research pipeline: - Smart dimension selection (Phase 1.5) with per-dimension relevance rationale - Methodology loading enforcement via explicit gating protocol in spawn prompts - Post-findings hard-fail gate on missing required frameworks with retry (max 2) - Report 9-section enforcement with "not assessed" placeholders and augment suggestions - Audience-tailored output (executives/pm/investors/dev/all) with section reordering - HTML output with inline CSS and Mermaid CDN rendering - Arg hint discoverability with enum values in all 4 skill frontmatter files - 10 autoresearch eval cases across 3 new eval directories - Custom dimension support with generic methodology fallback
Patch bump for methodology enforcement feature: - plugin.json: 0.5.0 → 0.5.1 - research-orchestrator: 0.5.0 → 0.5.1 - report-synthesizer: 0.1.0 → 0.1.1 - dimension-analyst: 0.4.0 → 0.4.1
There was a problem hiding this comment.
Pull request overview
This PR updates the Sigint research agents and skills to enforce methodology gating, introduce “smart” dimension selection, and standardize report generation (structure, audience tailoring, Mermaid visuals, HTML output), alongside adding autoresearch eval fixtures to validate these behaviors.
Changes:
- Adds Phase 1.5 dimension relevance/confirmation flow and strengthens methodology-gating + post-findings retry behavior in the research orchestrator.
- Introduces a 9-section report generation protocol with audience transforms and HTML output guidance in the report synthesizer.
- Updates skill frontmatter argument hints and adds new autoresearch eval suites (start/report/trend-analysis) with grading specs.
Reviewed changes
Copilot reviewed 21 out of 31 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/update/SKILL.md | Updates CLI argument hinting for dimensions in update mode. |
| skills/start/SKILL.md | Adds --dimensions hint/docs for start (preselect dimensions). |
| skills/report/SKILL.md | Expands argument-hint with enum values for format/audience/sections. |
| skills/augment/SKILL.md | Renames --methodology→--dimension and updates hints/docs. |
| agents/research-orchestrator.md | Adds Phase 1.5 selection flow; updates analyst spawn prompt to enforce methodology gating; adds Phase 2.75 hard-fail retry loop. |
| agents/report-synthesizer.md | Adds section iterator + audience transform + new Mermaid templates + HTML output guidance. |
| agents/dimension-analyst.md | Adds custom-dimension handling guidance in the methodology gating protocol. |
| skills/start-autoresearch/evals.json | Registers start autoresearch evals. |
| skills/start-autoresearch/iteration-0/eval-dimension-selection/grading.json | Grading spec for dimension selection UI/confirmation behavior. |
| skills/start-autoresearch/iteration-0/eval-dimension-selection/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/start-autoresearch/iteration-0/eval-methodology-loading/grading.json | Grading spec for analyst skill methodology loading/gating. |
| skills/start-autoresearch/iteration-0/eval-methodology-loading/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/start-autoresearch/iteration-0/eval-methodology-gate/grading.json | Grading spec for post-findings methodology hard-fail + retry. |
| skills/start-autoresearch/iteration-0/eval-methodology-gate/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/trend-analysis-autoresearch/evals.json | Registers trend-analysis autoresearch evals. |
| skills/trend-analysis-autoresearch/iteration-0/eval-mermaid-scenario/grading.json | Grading spec for trend scenario Mermaid generation. |
| skills/trend-analysis-autoresearch/iteration-0/eval-mermaid-scenario/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/trend-analysis-autoresearch/iteration-0/eval-trend-tables/grading.json | Grading spec for macro/micro trend tables + indicators. |
| skills/trend-analysis-autoresearch/iteration-0/eval-trend-tables/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/report-autoresearch/evals.json | Registers report autoresearch evals. |
| skills/report-autoresearch/iteration-0/eval-9-section-report/grading.json | Grading spec for 9-section enforcement + placeholders. |
| skills/report-autoresearch/iteration-0/eval-9-section-report/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/report-autoresearch/iteration-0/eval-audience-executive/grading.json | Grading spec for executive audience transforms + standalone summary output. |
| skills/report-autoresearch/iteration-0/eval-audience-executive/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/report-autoresearch/iteration-0/eval-html-format/grading.json | Grading spec for HTML output structure + inline CSS + Mermaid wrapping. |
| skills/report-autoresearch/iteration-0/eval-html-format/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/report-autoresearch/iteration-0/eval-positioning-map/grading.json | Grading spec for positioning map Mermaid conditions/content. |
| skills/report-autoresearch/iteration-0/eval-positioning-map/outputs/.gitkeep | Placeholder for eval outputs. |
| skills/report-autoresearch/iteration-0/eval-swot-mermaid/grading.json | Grading spec for SWOT Mermaid generation conditions/template. |
| skills/report-autoresearch/iteration-0/eval-swot-mermaid/outputs/.gitkeep | Placeholder for eval outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix arg-hint syntax: use list notation for comma-separated --dimensions flags - Add REQUESTED_DIMENSIONS passthrough from start skill to orchestrator - Add trend_modeling to augment dimension mapping table - Fix custom dimensions: store as strings in elicitation.dimensions, metadata in custom_dimensions - Add SKILL_OVERRIDE conditional to orchestrator spawn prompt for custom dimensions - Use schema-conformant methodology plan for custom dimensions in dimension-analyst - Add section→dimension mapping for augment placeholders in report synthesizer - Fix section id mismatch: exec-summary → executive-summary in audience tables - Remove non-existent section ids from audience override tables - Add multiple data points to Risk matrix Mermaid template - Add team_name to retry analyst spawn in Phase 2.75 Resolves review comments on PR #4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/sigint:augmentsuggestion — never silently omits sections--audience executives|pm|investors|dev|allwith section reordering, language register adjustment, and executive one-pager--format htmlproduces valid HTML5 with inline CSS and Mermaid CDN renderingFiles Changed
Modified (7):
agents/research-orchestrator.md— Phase 1.5 dimension selection, Phase 2.2 spawn prompt fix, Phase 2.75 methodology gateagents/report-synthesizer.md— Section Generation Protocol, Audience Transform Protocol, Porter's mindmap, Risk matrix, HTML templateagents/dimension-analyst.md— Custom dimension handling in Step 2skills/start/SKILL.md—--dimensionsarg-hint with enum valuesskills/report/SKILL.md—--format,--audience,--sectionsenum valuesskills/augment/SKILL.md— Renamed--methodologyto--dimensionwith enum valuesskills/update/SKILL.md—--dimensionsenum valuesCreated (23):
skills/start-autoresearch/— 3 eval cases (dimension-selection, methodology-loading, methodology-gate)skills/trend-analysis-autoresearch/— 2 eval cases (mermaid-scenario, trend-tables)skills/report-autoresearch/— 5 eval cases (9-section-report, audience-executive, swot-mermaid, positioning-map, html-format)Test Plan
/sigint:starton a test topic — verify dimension selection UI with rationale/sigint:report --audience executives --format markdown— verify exec-tailored output/sigint:report --format html— verify valid HTML with inline CSS/sigint:reportwith partial findings — verify "not assessed" placeholders with augment suggestions